Incorporating Concreteness in Multi-Modal Language Models with Curriculum Learning

Over the last few years, there has been an increase in studies that consider experiential (visual) information by building multi-modal language models and representations. It is shown several acquisition humans starts with learning concrete concepts through images then continues abstract ideas text. In this work, curriculum method used to teach model concrete/abstract their corresponding captions accomplish modeling/representation. We use BERT Resnet-152 on each modality combine them using attentive pooling perform pre-training newly constructed dataset, which collected from Wikimedia Commons based words. To show performance of proposed model, downstream tasks ablation are performed. The contribution work two-fold: A new dataset words, a approach proposed. results contributes success model.